Skip to content

update xpassing baggage tests for unimplemented languages#3773

Merged
rachelyangdog merged 9 commits intomainfrom
rachel.yang/xpassing-baggage-tests
Feb 6, 2025
Merged

update xpassing baggage tests for unimplemented languages#3773
rachelyangdog merged 9 commits intomainfrom
rachel.yang/xpassing-baggage-tests

Conversation

@rachelyangdog
Copy link
Contributor

@rachelyangdog rachelyangdog commented Jan 6, 2025

Motivation

Currently, some of the baggage tests are xpassing for langauges where baggage has not been implemented yet (this is because of the negative assertion tests).

Changes

Workflow

  1. ⚠️ Create your PR as draft ⚠️
  2. Work on you PR until the CI passes (if something not related to your task is failing, you can ignore it)
  3. Mark it as ready for review
    • Test logic is modified? -> Get a review from RFC owner. We're working on refining the codeowners file quickly.
    • Framework is modified, or non obvious usage of it -> get a review from R&P team

🚀 Once your PR is reviewed, you can merge it!

🛟 #apm-shared-testing 🛟

Reviewer checklist

  • If PR title starts with [<language>], double-check that only <language> is impacted by the change
  • No system-tests internal is modified. Otherwise, I have the approval from R&P team
  • CI is green, or failing jobs are not related to this change (and you are 100% sure about this statement)
  • A docker base image is modified?
    • the relevant build-XXX-image label is present
  • A scenario is added (or removed)?

@rachelyangdog rachelyangdog requested review from a team and mabdinur as code owners January 6, 2025 19:07
@rachelyangdog rachelyangdog changed the title xpassing baggage tests for unimplemented languages update xpassing baggage tests for unimplemented languages Jan 6, 2025
robertomonteromiguel
robertomonteromiguel requested changes Jan 7, 2025
View reviewed changes
bm1549
bm1549 reviewed Jan 7, 2025
View reviewed changes
Copy link
Contributor

@bm1549 bm1549 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[q] Is there any way we can make the same check for our baggage disabled tests (D003 and D007)?

@rachelyangdog
Copy link
Contributor Author

rachelyangdog commented Jan 15, 2025

[q] Is there any way we can make the same check for our baggage disabled tests (D003 and D007)?

I spoke with @cbeauchesne, and we concluded that the only way to pass the baggage_disable tests right now is to skip them for languages that haven’t implemented baggage yet. Since there’s no available telemetry or logs to differentiate which languages support baggage, I added a change to skip the tests for those languages. Let me know what you think!

cbeauchesne
cbeauchesne requested changes Jan 15, 2025
View reviewed changes
@rachelyangdog rachelyangdog merged commit ccc1c51 into main Feb 6, 2025
241 of 292 checks passed
@rachelyangdog rachelyangdog deleted the rachel.yang/xpassing-baggage-tests branch February 6, 2025 20:32
sezen-datadog added a commit that referenced this pull request Feb 10, 2025
@sezen-datadog
Extend vulnerability location data with class
41aa93f 
re-enable aws integrations tests (#3733)

feat(tests/test_data_integrity): update test_datadog_external_env for Go v1.72.0 and forward (#3961)

Activate ruff rules on tests/ folder (#3999)

[python] Avoid passing global tracer to pin in weblog apps (#4004)

All classes must declare feature ids (#4003)

Extend mypy scope (#4002)

Onboarding: bug marker profiling (#4005)

Docker SSI: fix scenario (#4006)

[ruby] Enable IP blocking tests for Ruby (#3937)

Activate ruff rules on tests/ folder (#4007)

[nodejs] remove auto login event skip (#3998)

Email HTML Injection detection in IAST Java (#3906)

Co-authored-by: Mario Vidal Domínguez <60353145+Mariovido@users.noreply.github.com>

Add test to check absence of client computed stats (#3812)

[java] Skip payara/CROSSED_TRACING_LIBRARIES/prod (#4009)

Add GraphQL error extension tests (#3986)

Co-authored-by: William Conti <william.conti@datadoghq.com>

Use prod agent, dev agent broke lot  of tracers (#4011)

update xpassing baggage tests for unimplemented languages (#3773)

[NodeJS] skip failing baggage tests (#4015)

[python] fix 500 errors in sql queries (#3997)

Activate ruff rules on tests/ folder (#4010)

Hotfix

Fix fuzzer

[Nodejs] Enable untrusted deserialization stack trace test for Node.js (#3995)

[python] use main again for dev branch (#4008)

Co-authored-by: erikayasuda <153395705+erikayasuda@users.noreply.github.com>
Co-authored-by: Charles de Beauchesne <charles.debeauchesne@datadoghq.com>

Revert agent dev fix (#4013)

[PHP] Enable rasp telemetry tests for PHP (#3972)

[skipci] Update CODEOWNERS for static files (#4012)

[Java] Enable more easy wins (#4018)

[java] Bump GraalVM system test to JDK 22 (#4001)

[NodeJS] skip more failing baggage tests (#4021)

[Debugger] Update dotnet Exception Replay tests (#3974)

Test multiple rasp during one request (#3989)

Add test for location extended data (#3978)

Fix APPSEC_NO_STATS scenario name (#4019)

Avoid false XPASS on APPSEC_WAF_TELEMETRY (#4029)

[java] Enable Test_Blocking_strip_response_headers in some variants (#4033)

[java] Remove some outdated manifest entries (#4039)

[java] Fix xpass for Test_SecurityEvents_Appsec_Metastruct_Disabled (#4038)

Consolidate remote config tests into same directory/file (#4031)

[python] use last patch version of python for django weblogs (#4025)

crashtracking: assert si_signo is set to 11 (#4023)
sezen-datadog added a commit that referenced this pull request Feb 10, 2025
@sezen-datadog
# This is the 1st commit message:
d2064eb 
Extend vulnerability location data with class

re-enable aws integrations tests (#3733)

feat(tests/test_data_integrity): update test_datadog_external_env for Go v1.72.0 and forward (#3961)

Activate ruff rules on tests/ folder (#3999)

[python] Avoid passing global tracer to pin in weblog apps (#4004)

All classes must declare feature ids (#4003)

Extend mypy scope (#4002)

Onboarding: bug marker profiling (#4005)

Docker SSI: fix scenario (#4006)

[ruby] Enable IP blocking tests for Ruby (#3937)

Activate ruff rules on tests/ folder (#4007)

[nodejs] remove auto login event skip (#3998)

Email HTML Injection detection in IAST Java (#3906)

Co-authored-by: Mario Vidal Domínguez <60353145+Mariovido@users.noreply.github.com>

Add test to check absence of client computed stats (#3812)

[java] Skip payara/CROSSED_TRACING_LIBRARIES/prod (#4009)

Add GraphQL error extension tests (#3986)

Co-authored-by: William Conti <william.conti@datadoghq.com>

Use prod agent, dev agent broke lot  of tracers (#4011)

update xpassing baggage tests for unimplemented languages (#3773)

[NodeJS] skip failing baggage tests (#4015)

[python] fix 500 errors in sql queries (#3997)

Activate ruff rules on tests/ folder (#4010)

Hotfix

Fix fuzzer

[Nodejs] Enable untrusted deserialization stack trace test for Node.js (#3995)

[python] use main again for dev branch (#4008)

Co-authored-by: erikayasuda <153395705+erikayasuda@users.noreply.github.com>
Co-authored-by: Charles de Beauchesne <charles.debeauchesne@datadoghq.com>

Revert agent dev fix (#4013)

[PHP] Enable rasp telemetry tests for PHP (#3972)

[skipci] Update CODEOWNERS for static files (#4012)

[Java] Enable more easy wins (#4018)

[java] Bump GraalVM system test to JDK 22 (#4001)

[NodeJS] skip more failing baggage tests (#4021)

[Debugger] Update dotnet Exception Replay tests (#3974)

Test multiple rasp during one request (#3989)

Add test for location extended data (#3978)

Fix APPSEC_NO_STATS scenario name (#4019)

Avoid false XPASS on APPSEC_WAF_TELEMETRY (#4029)

[java] Enable Test_Blocking_strip_response_headers in some variants (#4033)

[java] Remove some outdated manifest entries (#4039)

[java] Fix xpass for Test_SecurityEvents_Appsec_Metastruct_Disabled (#4038)

Consolidate remote config tests into same directory/file (#4031)

[python] use last patch version of python for django weblogs (#4025)

crashtracking: assert si_signo is set to 11 (#4023)

# This is the commit message #2:

[nodejs] Fix get_target_branch - "quotes, parens and # (#3952)"" (#3953)

Co-authored-by: Charles de Beauchesne <charles.debeauchesne@datadoghq.com>
# This is the commit message #3:

Fix Standalone Billing values (#3965)

# This is the commit message #4:

Add success to valid WAF telemetry tags (#3966)

# This is the commit message #5:

Ensure tracer release coherence (#3967)

[Nodejs] Code injection telemetry metric (#3959)

Do not use special chars in "redacted" string used in scrubber (#3977)

fix Context Propagation: Extraction feature_id (#3970)

[dotnet] Enable Custom Blocking Response tests (#3971)

[python] update flask base image (#3979)

[ruby] Skip failing test for APMAPI-1141 (#3980)

update weblog sampling_test use sampling rules instead of deprecated envvar (#3984)

[python] enable Python IAST+SSRF tests (#2512)

Signed-off-by: Juanjo Alvarez <juanjo.alvarezmartinez@datadoghq.com>

[python] IAST Header source in werkzeug 3.1 (#3991)

[python] switch flaky decorator for bug (#3990)

[python] update versions for major release (#3993)

[test optimization] Update config shape sent as telemetry (#3992)

Add profiling scenarios to Python onboarding tests (#3002)

Update changelog

re-enable aws integrations tests (#3733)

feat(tests/test_data_integrity): update test_datadog_external_env for Go v1.72.0 and forward (#3961)

[python] Avoid passing global tracer to pin in weblog apps (#4004)

All classes must declare feature ids (#4003)

Extend mypy scope (#4002)

Onboarding: bug marker profiling (#4005)

Docker SSI: fix scenario (#4006)

[ruby] Enable IP blocking tests for Ruby (#3937)

[nodejs] remove auto login event skip (#3998)

[NodeJS] skip failing baggage tests (#4015)

[python] fix 500 errors in sql queries (#3997)

Hotfix

Fix fuzzer

[python] use main again for dev branch (#4008)

Co-authored-by: erikayasuda <153395705+erikayasuda@users.noreply.github.com>
Co-authored-by: Charles de Beauchesne <charles.debeauchesne@datadoghq.com>

Revert agent dev fix (#4013)

[skipci] Update CODEOWNERS for static files (#4012)

[Java] Enable more easy wins (#4018)

[java] Bump GraalVM system test to JDK 22 (#4001)

[NodeJS] skip more failing baggage tests (#4021)

[Debugger] Update dotnet Exception Replay tests (#3974)

Test multiple rasp during one request (#3989)

Add test for location extended data (#3978)

Fix APPSEC_NO_STATS scenario name (#4019)

Avoid false XPASS on APPSEC_WAF_TELEMETRY (#4029)

[java] Enable Test_Blocking_strip_response_headers in some variants (#4033)

[java] Remove some outdated manifest entries (#4039)

[java] Fix xpass for Test_SecurityEvents_Appsec_Metastruct_Disabled (#4038)

Consolidate remote config tests into same directory/file (#4031)

[python] use last patch version of python for django weblogs (#4025)

crashtracking: assert si_signo is set to 11 (#4023)

class vs path
sezen-datadog added a commit that referenced this pull request Feb 10, 2025
@sezen-datadog
# This is the 1st commit message:
d4259f6 
Extend vulnerability location data with class

re-enable aws integrations tests (#3733)

feat(tests/test_data_integrity): update test_datadog_external_env for Go v1.72.0 and forward (#3961)

Activate ruff rules on tests/ folder (#3999)

[python] Avoid passing global tracer to pin in weblog apps (#4004)

All classes must declare feature ids (#4003)

Extend mypy scope (#4002)

Onboarding: bug marker profiling (#4005)

Docker SSI: fix scenario (#4006)

[ruby] Enable IP blocking tests for Ruby (#3937)

Activate ruff rules on tests/ folder (#4007)

[nodejs] remove auto login event skip (#3998)

Email HTML Injection detection in IAST Java (#3906)

Co-authored-by: Mario Vidal Domínguez <60353145+Mariovido@users.noreply.github.com>

Add test to check absence of client computed stats (#3812)

[java] Skip payara/CROSSED_TRACING_LIBRARIES/prod (#4009)

Add GraphQL error extension tests (#3986)

Co-authored-by: William Conti <william.conti@datadoghq.com>

Use prod agent, dev agent broke lot  of tracers (#4011)

update xpassing baggage tests for unimplemented languages (#3773)

[NodeJS] skip failing baggage tests (#4015)

[python] fix 500 errors in sql queries (#3997)

Activate ruff rules on tests/ folder (#4010)

Hotfix

Fix fuzzer

[Nodejs] Enable untrusted deserialization stack trace test for Node.js (#3995)

[python] use main again for dev branch (#4008)

Co-authored-by: erikayasuda <153395705+erikayasuda@users.noreply.github.com>
Co-authored-by: Charles de Beauchesne <charles.debeauchesne@datadoghq.com>

Revert agent dev fix (#4013)

[PHP] Enable rasp telemetry tests for PHP (#3972)

[skipci] Update CODEOWNERS for static files (#4012)

[Java] Enable more easy wins (#4018)

[java] Bump GraalVM system test to JDK 22 (#4001)

[NodeJS] skip more failing baggage tests (#4021)

[Debugger] Update dotnet Exception Replay tests (#3974)

Test multiple rasp during one request (#3989)

Add test for location extended data (#3978)

Fix APPSEC_NO_STATS scenario name (#4019)

Avoid false XPASS on APPSEC_WAF_TELEMETRY (#4029)

[java] Enable Test_Blocking_strip_response_headers in some variants (#4033)

[java] Remove some outdated manifest entries (#4039)

[java] Fix xpass for Test_SecurityEvents_Appsec_Metastruct_Disabled (#4038)

Consolidate remote config tests into same directory/file (#4031)

[python] use last patch version of python for django weblogs (#4025)

crashtracking: assert si_signo is set to 11 (#4023)

# This is the commit message #2:

[nodejs] Fix get_target_branch - "quotes, parens and # (#3952)"" (#3953)

Co-authored-by: Charles de Beauchesne <charles.debeauchesne@datadoghq.com>
# This is the commit message #3:

Fix Standalone Billing values (#3965)

# This is the commit message #4:

Add success to valid WAF telemetry tags (#3966)

# This is the commit message #5:

Ensure tracer release coherence (#3967)

[Nodejs] Code injection telemetry metric (#3959)

Do not use special chars in "redacted" string used in scrubber (#3977)

fix Context Propagation: Extraction feature_id (#3970)

[dotnet] Enable Custom Blocking Response tests (#3971)

[python] update flask base image (#3979)

[ruby] Skip failing test for APMAPI-1141 (#3980)

update weblog sampling_test use sampling rules instead of deprecated envvar (#3984)

[python] enable Python IAST+SSRF tests (#2512)

Signed-off-by: Juanjo Alvarez <juanjo.alvarezmartinez@datadoghq.com>

[python] IAST Header source in werkzeug 3.1 (#3991)

[python] switch flaky decorator for bug (#3990)

[python] update versions for major release (#3993)

[test optimization] Update config shape sent as telemetry (#3992)

Add profiling scenarios to Python onboarding tests (#3002)

Update changelog

re-enable aws integrations tests (#3733)

feat(tests/test_data_integrity): update test_datadog_external_env for Go v1.72.0 and forward (#3961)

[python] Avoid passing global tracer to pin in weblog apps (#4004)

All classes must declare feature ids (#4003)

Extend mypy scope (#4002)

Onboarding: bug marker profiling (#4005)

Docker SSI: fix scenario (#4006)

[ruby] Enable IP blocking tests for Ruby (#3937)

[nodejs] remove auto login event skip (#3998)

[NodeJS] skip failing baggage tests (#4015)

[python] fix 500 errors in sql queries (#3997)

Hotfix

Fix fuzzer

[python] use main again for dev branch (#4008)

Co-authored-by: erikayasuda <153395705+erikayasuda@users.noreply.github.com>
Co-authored-by: Charles de Beauchesne <charles.debeauchesne@datadoghq.com>

Revert agent dev fix (#4013)

[skipci] Update CODEOWNERS for static files (#4012)

[Java] Enable more easy wins (#4018)

[java] Bump GraalVM system test to JDK 22 (#4001)

[NodeJS] skip more failing baggage tests (#4021)

[Debugger] Update dotnet Exception Replay tests (#3974)

Test multiple rasp during one request (#3989)

Add test for location extended data (#3978)

Fix APPSEC_NO_STATS scenario name (#4019)

Avoid false XPASS on APPSEC_WAF_TELEMETRY (#4029)

[java] Enable Test_Blocking_strip_response_headers in some variants (#4033)

[java] Remove some outdated manifest entries (#4039)

[java] Fix xpass for Test_SecurityEvents_Appsec_Metastruct_Disabled (#4038)

Consolidate remote config tests into same directory/file (#4031)

[python] use last patch version of python for django weblogs (#4025)

crashtracking: assert si_signo is set to 11 (#4023)

class vs path
robertomonteromiguel pushed a commit that referenced this pull request Mar 17, 2025
@rachelyangdog @robertomonteromiguel
update xpassing baggage tests for unimplemented languages (#3773)
5e534ba
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@cbeauchesne cbeauchesne cbeauchesne approved these changes

@bm1549 bm1549 bm1549 approved these changes

@mabdinur mabdinur mabdinur approved these changes

@robertomonteromiguel robertomonteromiguel robertomonteromiguel approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@rachelyangdog @cbeauchesne @bm1549 @mabdinur @robertomonteromiguel